Combining Textual and Visual Thesaurus for a Multi-modal Search in a Satellite Image Database

Satellite images are more and more numerous and with increasing resolution. Hence it is urgent to develop tools able to (semi-) automatically process such images in order to efficiently exploit them. When searching satellite images by visual content, in most of the cases the query image resides in the mind of the user as a set of subjective visual patterns, psychological impressions or “mental pictures”. In this paper, we enrich the visual description by introducing to the user “the page zero” which is multiple visual patches summarizing the image database. Since there is no perfect description of visual content of images, most methods try to find a good compromise in balancing the image description between low and height level features. However, there exists an evident semantic gap between the demanding of user and the representation of low-level features [6]. Text-based methods are very powerful in matching context but do not have access to image content. The basic idea of our work is to combine high level and low level features to find a class of ‘similar’ images with similar keywords and with similar descriptors. In this paper, a novel content-based image retrieval framework is introduced. We propose three retrieval strategies by typing a keyword, selecting a visual thesaurus or by using the multi-modal description. We can prove that the multi-modal approach is able to retrieve complex concepts better than standards visual or semantical approaches taken separately.